High-level Control of Autonomous Robots Using a Behavior-based Scheme and Reinforcement Learning

This paper proposes a behavior-based scheme for high-level control of autonomous robots. Two main characteristics can be highlighted in the control scheme. Behavior coordination is done through a hybrid methodology, which takes in advantages of the robustness and modularity in competitive approaches, as well as optimized trajectories in cooperative ones. As a second feature, behavior state/action mapping is learnt by means of Reinforcement Learning (RL). A new continuous approach of the Q_learning algorithm, implemented with a multi-layer neural network, is used. The behavior-based scheme attempts to fulfill simple missions in which several behaviors/tasks compete for the vehicle’s control. This paper is centered in the RL-based behaviors. In order to test the feasibility of the proposed Neural-Q_learning scheme, real experiments with the underwater robot ODIN in a target following behavior were done. Results showed the convergence of the behavior into an optimal state/action mapping. Discussion about the proposed approach is given, as well as an overall description of the high level control scheme. Copyright  2002 IFAC